Cooperative Sensation: A Mechanism For Robust Human-Computer Interaction

The benefits of a network of cooperative multi-modal sensors are presented in the contexts of three different human-computer interaction systems. First, a robust automatic teleconferencing system with multiple microphones and cameras is used to illustrate the benefits of multi-modal sensor fusion. The integrated speaker localization system doubles the original accuracy of the individual sensors and functions correctly at signal-to-noise ratios as low as 0.5 dB. Next, multiple microphones are used to improve speaker independent speech recognition, enabling a word recognition rate of approximately 90% at signal-to-noise ratios as low as –20dB. Finally, several dynamic microphone arrays (DMA), which, when attached to a Personal Digital Assistant (PDA) or other appropriate processing device, can roam freely in the environment, use the active speakers in the environment as beacons in order to localize and track their own locations and that of their corresponding PDA. Key-Words: human computer interaction, sensor fusion, multi-modal perception